ParsBERT: Transformer-based Model for Persian Language Understanding

The surge of pre-trained language models has begun a new era in the field Natural Language Processing (NLP) by allowing us to build powerful models. Among these models, Transformer-based such as BERT have become increasingly popular due their state-of-the-art performance. However, are usually focused on English, leaving other languages multilingual with limited resources. This paper proposes monolingual for Persian (ParsBERT), which shows its performance compared architectures and Also, since amount data available NLP tasks is very restricted, massive dataset different well pre-training model composed. ParsBERT obtains higher scores all datasets, including existing ones gathered ones, improves outperforming both prior works Sentiment Analysis, Text Classification, Named Entity Recognition tasks.